That's what PGP Keys were made for.